Assessing ChatGPT 4.0's test performance and clinical diagnostic accuracy on USMLE STEP 2 CK and clinical case reports.

While there is data assessing the test performance of artificial intelligence (AI) chatbots, including the Generative Pre-trained Transformer 4.0 (GPT 4) chatbot (ChatGPT 4.0), there is scarce data on its diagnostic accuracy of clinical cases. We assessed the large language model (LLM), ChatGPT 4.0, on its ability to answer questions from the United States Medical Licensing Exam (USMLE) Step 2, as well as its ability to generate a differential diagnosis based on corresponding clinical vignettes from published case reports. A total of 109 Step 2 Clinical Knowledge (CK) practice questions were inputted into both ChatGPT 3.5 and ChatGPT 4.0, asking ChatGPT to pick the correct answer. Compared to its previous version, ChatGPT 3.5, we found improved accuracy of ChatGPT 4.0 when answering these questions, from 47.7 to 87.2% (p = 0.035) respectively. Utilizing the topics tested on Step 2 CK questions, we additionally found 63 corresponding published case report vignettes and asked ChatGPT 4.0 to come up with its top three differential diagnosis. ChatGPT 4.0 accurately created a shortlist of differential diagnoses in 74.6% of the 63 case reports (74.6%). We analyzed ChatGPT 4.0's confidence in its diagnosis by asking it to rank its top three differentials from most to least likely. Out of the 47 correct diagnoses, 33 were the first (70.2%) on the differential diagnosis list, 11 were second (23.4%), and three were third (6.4%). Our study shows the continued iterative improvement in ChatGPT's ability to answer standardized USMLE questions accurately and provides insights into ChatGPT's clinical diagnostic accuracy.

Rethinking the consultation paradigm: Validity evidence for a new framework, a multimethods study.

BACKGROUND: In-hospital consultation is essential for patient care. We previously proposed a framework of seven specific consultation types to classify consult requests to improve communication, workflow, and provider satisfaction. METHODS: This multimethods study's aim was to evaluate the applicability of the consult classification framework to real internal medicine (IM) consults. We sought validity evidence using Kane's validity model with focus groups and classifying consult requests from five IM specialties. Participants attended five 1 h semi-structured focus groups that were recorded, transcribed, and coded for thematic saturation. For each specialty, three specialists and three hospitalists categorized 100 (total 500) random anonymized consult requests. The primary outcome was concordance in the classification of consult requests, defined as the sum of partial concordance and perfect concordance, where respectively 4-5/6 and 6/6 participants classified a consult in the same category. We used χ2 tests to compare concordance rates across specialties and between specialists and hospitalists. RESULTS: Five major themes were identified in the qualitative analysis of the focus groups: (1) consult question, (2) interpersonal interactions, (3) value, (4) miscommunication, (5) consult framework application, barriers, and iterative development. In the quantitative analysis, the overall concordance rate was 88.8% (95% confidence interval [CI]: 85.7-91.4), and perfect concordance was 46.6% (95% CI: 42.2-51.1). Concordance differed significantly between hospitalists and specialists overall (p = .01), with a higher proportion of hospitalists having perfect concordance compared to specialists (67.2% vs. 57.8%, p = .002). CONCLUSIONS: The consult classification framework was found to be applicable to consults from five different IM specialties, and could improve communication and education.

A Predictive Algorithm for Discriminating Myeloid Malignancies and Leukemoid Reactions.

BACKGROUND: Adults presenting with a neutrophil-predominant leukocytosis (white cell count >50,000/µL) often necessitate urgent medical management. These patients are diagnosed with either acute presentations of chronic myeloid malignancies or leukemoid reactions, yet accurate models to distinguish between these entities do not exist. We used demographic and lab data to build a machine learning model capable of discriminating between these diagnoses. METHODS: The medical record at a tertiary care medical center was queried to identify adults with instances of white counts greater than 50,000/µL and >50% neutrophils from 2000-2021. For each patient, a full set of demographic and lab values were extracted at the time of their first presentation with a white count >50,000/µL. We generated a series of models in which the parameters most predictive of myeloid malignancies were identified, and a supervised machine learning approach was applied to the dataset. RESULTS: Our best model - using a support vector machine algorithm - produced a sensitivity of 96.0% and a specificity of 95.9% (area under the curve = 0.982) for identifying myeloid malignancies. We also identified a clinically meaningful and significant disparity in outcomes based on diagnosis - a six-fold increase in 12-month mortality in those diagnosed with leukemoid reactions. CONCLUSIONS: These findings need to be validated but fill an unmet need for timely and accurate diagnosis in the setting of profound, neutrophil-predominant leukocytosis and support the use of predictive models as a means to improve patient outcomes.

Novel interactive text-messaging curriculum for endocrinology board review.

Background: The ABIM certification exam is one of the measures to ensure that physicians have the clinical skills for good care delivery. The 5-year average pass rate for ABIM Endocrinology exam is 82%. The pass rate significantly decreased to a nadir of 74% in 2021 and 2022, lowest of all medicine subspecialties. Objectives: To assess the feasibility of text messaging curriculum for fellows and its utility in improving their test performance. Methods: In 2021, endocrinology fellows from 51 programs across the country were invited to participate in our curriculum. They completed a pre-test, joined a texting group via Remind application and received 1 multiple choice question daily (total n = 78). After 15 weeks, they completed a post-test and survey. Paired results from pre- and post-test were compared. Results: A total of 89 fellows from 27 programs responded. Of these, 82 fellows, predominantly females (n = 60; 73 %), filled out the pre-test. On an average, 42 fellows (SD = 12) responded to the questions daily and 57 % of them answered the questions within 24 h. Thirty fellows completed the post-test. The median number of correct responses on the pre-test was 5 (IQR 3-6), compared to 8 (IQR 6-9) in the post-test. There was a significant improvement (p-value < 0.0001) in fellows' performance in the post-test when compared with the pre-test following our intervention. Conclusions: Text-messaging based curriculum for exam preparation is feasible and can improve test performance. Fellows find receiving a daily high yield multiple choice question via text-message as a useful tool for exam preparation.

Consensus recommendations on peripheral blood smear review: defining curricular standards and fellow competency.

Graduate medical education training in hematology in North America is accredited by the Accreditation Council for Graduate Medical Education (ACGME). Trainees routinely review peripheral blood smears (PBS) in providing clinical care. Competency in PBS review at graduation is required by the ACGME. However, there are no consensus guidelines on best practices surrounding PBS review, education, or competency. We describe the generation of proposed theory and the consensus recommendations developed through a multi-institutional focus group, developed using constructivist grounded theory and a modified nominal group technique. Eight academic hematologists, spanning classical and malignant hematology, enrolled and participated in 2 one-hour focus groups. All routinely worked with fellows and half had formally instructed trainees on PBS interpretation. Focus group data were analyzed using mixed-methods techniques. Tenets of emerging theory were identified through inductive coding. Consensus recommendations (CR) were generated. Participants reviewed CR in an iterative fashion until consensus was reached. Strong consensus was reached on multiple aspects of PBS education. All agreed that trainees should learn PBS review through a systematic approach. Group discussion focused on disorders of red and white blood cells. The diagnoses of acute leukemia and thrombotic microangiopathies were most commonly discussed, with specific emphasis on disorders in which prompt recognition was required to avert significant patient morbidity. These CR offer external validity to future research and curricular development for both PBS review and other visuospatial tasks in medical education.

A Resident-as-Leader Curriculum for Managing Inpatient Teams.

PROBLEM: Leading inpatient teams is a foundational clinical responsibility of resident physicians and leadership is a core competency for inpatient physicians, yet few training programs have formal leadership curricula to realize this clinical skill. INTERVENTION: We implemented a 4-module curriculum for PGY1 internal medicine residents. The program focused on the managerial skills necessary for daily clinical leadership, followed by clinical coaching. Interns were first introduced to foundational concepts and then given the opportunity to apply those concepts to real-world practice followed by clinical coaching. CONTEXT: Using direct-observations and a previously published checklist for rounds leadership, this study sought to evaluate the workplace behavior change for novice residents leading inpatient teams for the first time. We conducted a prospective cohort study (March 2016 and August 2018) of internal medicine residents at a large tertiary academic medical center in Boston, MA. Trained faculty raters performed direct observations of clinical rounding experiences using the checklist and compared the findings to historical and internal controls. Questionnaires were distributed pre- and post- curriculum to assess satisfaction and readiness to lead a team. IMPACT: We trained 65 PGY1 residents and raters conducted 140 direct observations - 36 in the intervention group and 104 among historical controls. The unadjusted mean score in rounds leadership skills for the intervention group was 19.0 (SD = 5.1) compared to 16.2 (SD = 6.2) for historical controls. Adjusting for repeated measures, we found significant improvement in mean scores for behaviors linked to the curricular objectives (p = 0.008) but not for general behaviors not covered by the curriculum (p = 0.2). LESSONS LEARNED: A formal curriculum to train residents as leaders led to behavior change in the workplace in domains essential to rounds leadership. We also found that the curriculum was highly regarded in that all interns indicated they would recommend the curriculum to a peer. Moreover, the program may have assuaged some anxiety during the transition to junior year as 90% of interns surveyed felt more ready to start PGY2 year than historical trainings. We learned that while a robust, multi-faceted modular curriculum and clinical coaching successfully resulted in behavior change, the resources required to manage this program are significant and difficult to sustain. Future iterations could include asynchronous material and potentially peer-observation of rounds leadership to reduce the burden on faculty and program curricular time.

Implementing a Virtual Flipped Classroom in a Rheumatology Fellowship Program.

OBJECTIVE: Active learning opportunities within graduate medical education may be underused. We aimed to assess whether active learning strategies increase after implementing a faculty development workshop and transitioning rheumatology fellowship didactics to a virtual flipped classroom. METHODS: We measured baseline faculty use of active learning strategies during lectures within the Introductory Rheumatology Curriculum by calculating an "active learning score" from a cognitive learning theory assessment tool. We held a faculty development workshop demonstrating active teaching strategies and encouraged using a flipped classroom for fellowship didactics. Because of the COVID-19 pandemic, the strategies were discussed in a virtual classroom setting where the intervention phase would occur. We compared active learning scores before and after the intervention for lectures within the Introductory Rheumatology Curriculum. The primary outcome was the change in active teaching scores preintervention versus postintervention. RESULTS: Active learning scores increased in 14 of the 16 lectures, with a mean score increase of 4.7 of 24 points (95% confidence interval 2.3-7.2). Paired t-test analyses comparing active learning scores preintervention and postintervention for each lecture confirmed that results were highly statistically significant (P < 0.001). Despite faculty hesitancy to teach within a virtual environment, faculty satisfaction remained high postintervention. CONCLUSION: A virtual flipped classroom increased the use of active learning strategies within the Introductory Rheumatology Curriculum. Faculty satisfaction remained high despite modest increases in time spent updating their presentations. Fellows and faculty reported a largely positive experience within the virtual classroom.

How Would You Treat This Patient With Pulmonary Embolism? : Grand Rounds Discussion From Beth Israel Deaconess Medical Center.

Pulmonary embolism can be acutely life-threatening and is associated with long-term consequences such as recurrent venous thromboembolism and chronic thromboembolic pulmonary hypertension. In 2020, the American Society of Hematology published updated guidelines on the management of patients with venous thromboembolism. Here, a hematologist and a cardiology and vascular medicine specialist discuss these guidelines in the context of the care of a patient with pulmonary embolism. They discuss advanced therapies such as catheter-directed thrombolysis in the short-term management of patients with intermediate-risk disease, recurrence risk stratification at presentation, and ideal antithrombotic regimens for patients whose pulmonary embolism was associated with a transient minor risk factor.

Oncology Fellows' Clinical Discussions, Perceived Knowledge, and Formal Training Regarding Medical Cannabis Use: A National Survey Study.

PURPOSE: Evidence suggests that patients with cancer frequently use cannabis with medicinal intent and desire clinical guidance from providers. We aimed to determine whether oncology training adequately prepares fellows to discuss medical cannabis. METHODS: A national survey study was conducted from January to March 2021. A questionnaire assessing oncology fellows' practices regarding cannabis recommendations in cancer care and their knowledge of its effectiveness and risks compared with conventional care for cancer-related symptoms was developed and sent to 155 US-based oncology training programs to distribute to trainees. RESULTS: Forty programs from 25 states participated; of the 462 trainees across these programs, 189 responded (response rate of 40%). Of the participants, 52% were female; 52% were White, 33% Asian, and 5% Hispanic. Fifty-seven percent reported that they discussed medical cannabis with more than five patients in the preceding year; however, only 13% felt sufficiently informed to issue cannabis-related clinical recommendations. Twenty-four percent reported having received formal training regarding medical cannabis. Oncology fellows who reported having received prior training in medical cannabis were significantly more likely to discuss cannabis with patients (risk ratio: 1.37, 95% CI 1.06 to 1.75; P = .002) and feel sufficiently informed to discuss cannabis recommendations (risk ratio: 5.06; 95% CI, 2.33 to 10.99; P < .001). Many viewed the botanical as a useful adjunctive therapy that was at least as effective as conventional treatments for anorexia/cachexia (72%), nausea/vomiting (45%), and pain (41%). CONCLUSION: Most oncology trainees not only reported engaging in discussions regarding medical cannabis with patients but also considered themselves insufficiently informed to make cannabis-related clinical recommendations. The discrepancy between the frequency of cannabis inquiries/discussions at the patient level and comfort/knowledge at the trainee provider level represents an unmet curricular need with implications for both graduate medical education and patient care.

Engaging trainees within the virtual classroom: Teaching strategies for rheumatologists in a pandemic.

Social distancing during the COVID-19 pandemic has led to unprecedented challenges in medical education, including for rheumatology training programs. Many programs have adapted by transitioning educational curricula into virtual classrooms. Herein, we review strategies to optimize learning within the virtual classroom. We introduce the flipped virtual classroom as a framework for facilitating higher-order thinking and improving long-term learning. We provide recommendations to maximize interactions between learners, elevate group discussions, and encourage problem solving. Once implemented, these techniques can lead to more productive teaching and learning experiences while maintaining a sense of community for rheumatology training programs.

More Specialties, Fewer Problems: Using Collaborative Competency Between Infectious Diseases, Podiatry, and Pathology to Improve the Care of Patients with Diabetic Foot Osteomyelitis.

BACKGROUND: Diabetic foot osteomyelitis is a common infection where treatment involves multiple services, including infectious diseases, podiatry, and pathology. Despite its ubiquity in the hospital, consensus on much of its management is lacking. METHODS: Representatives from infectious diseases, podiatry, and pathology interested in quality improvement developed multidisciplinary institutional recommendations culminating in an educational intervention describing optimal diagnostic and therapeutic approaches to diabetic foot osteomyelitis (DFO). Knowledge acquisition was assessed by preintervention and postintervention surveys. Inpatients with forefoot DFO were retrospectively reviewed before and after intervention to assess frequency of recommended diagnostic and therapeutic maneuvers, including appropriate definition of surgical bone margins, definitive histopathology reports, and unnecessary intravenous antibiotics or prolonged antibiotic courses. RESULTS: A postintervention survey revealed significant improvements in knowledge of antibiotic treatment duration and the role of oral antibiotics in managing DFO. There were 104 consecutive patients in the preintervention cohort (April 1, 2018, to April 1, 2019) and 32 patients in the postintervention cohort (November 5, 2019, to March 1, 2020), the latter truncated by changes in hospital practice during the coronavirus disease 2019 pandemic. Noncategorizable or equivocal disease reports decreased from before intervention to after intervention (27.0% versus 3.3%, respectively; P = .006). We observed nonsignificant improvement in correct bone margin definition (74.0% versus 87.5%; P = .11), unnecessary peripherally inserted central catheter line placement (18.3% versus 9.4%; P = .23), and unnecessary prolonged antibiotics (21.9% versus 5.0%; P = .10). In addition, by working as an interdisciplinary group, many solvable misunderstandings were identified, and processes were adjusted to improve the quality of care provided to these patients. CONCLUSIONS: This quality improvement initiative regarding management of DFO led to improved provider knowledge and collaborative competency between these three departments, improvements in definitive pathology reports, and nonsignificant improvement in several other clinical endpoints. Creating collaborative competency may be an effective local strategy to improve knowledge of diabetic foot infection and may generalize to other common multidisciplinary conditions.

Invasive Aspergillus flavus sinusitis in an immunocompetent patient using intranasal cocaine.

Invasive fungal sinusitis is a rare and potentially fatal infection that tends to occur in immunocompromised hosts. Presented is the case of a 33-year-old immunocompetent male with several months of recurrent facial and nasal pain refractory to several antibacterial courses before a diagnosis of invasive Aspergillus sinusitis was made. The patient's symptoms and infection were successfully treated with a combination of surgical debridement and voriconazole. The authors review the epidemiology, risk factors, diagnosis, and treatment of invasive fungal sinusitis due to Aspergillus.

Generalists as Clinical Physiologists: Bringing Science Back to the Bedside.

BACKGROUND: Few generalists engage in basic science research or feel comfortable teaching physiology at the bedside. This may reflect a lack of understanding or confidence teaching physiologic principles. AIM: To inspire general internists to relearn and teach physiology in clinical practice. SETTING: An active biomedical research laboratory. PARTICIPANTS: We educated 67 faculty participants (4 primary care, 59 hospitalists, and 4 other specialties) from 24 medical centers, representing 17 states. PROGRAM DESCRIPTION: The 5-day course was structured around re-learning basic physiology principles and developing teaching skills. Participants engaged in hands-on experiments through 4 modules using aquatic species, each paired with a physiology content primer. Participants also developed teaching scripts based on their experiments. PROGRAM EVALUATION: Post-course surveys revealed that 97% felt confident teaching physiology at the bedside, 100% felt the course enhanced their understanding of the mechanisms of disease, and there was a significant improvement in self-reported teaching ability. DISCUSSION: An immersive, hands-on faculty development course that integrated physiology with clinical decision-making increased participants' comfort level and self-rated ability to teach and incorporate physiology in their clinical work. We believe faculty development is one potential solution to the growing chasm between clinicians and scientists in general medicine.

Incidence and risk factors for PTT prolongation in patients receiving low-dose unfractionated heparin thromboprophylaxis.

Low-dose unfractionated heparin (LDUH) prophylaxis decreases the incidence of venous thromboembolism (VTE) in hospitalized patients, but increases the risk of bleeding events. Patients who develop a prolonged activated partial thromboplastin time (aPTT) while on LDUH may be at higher risk for bleeding complications. To determine the incidence and risk factors for aPTT prolongation in hospitalized patients receiving LDUH thromboprophylaxis, we performed a retrospective pharmacoepidemiologic cohort study of adult patients admitted to an academic medical center from September 2013 through September 2015. Among 3857 patients with at least one aPTT checked within 24 h of LDUH administration, aPTT prolongation > 1.5 times the upper limit of normal occurred in 131 (3.4%). Age 68-78 years (OR 1.6, 95% CI 1.01-2.4), age > 78 years (OR 1.9, 95% CI 1.3-2.9), female gender (OR 1.9, 95% CI 1.4-2.5), black race (OR 1.6, 95% CI 1.1-2.3), low BMI (OR 1.8, 95% CI 1.3-2.5), being admitted to a surgical service (OR 0.5, 95% CI 0.3-0.8), and receipt of high-dose (> 10,000 units in a day) unfractionated heparin prophylaxis (OR 1.4, 95% CI 1.003-2.0), were independently associated with aPTT prolongation after LDUH exposure. LDUH VTE prophylaxis is associated with aPTT prolongation in 3.4% of general hospitalized patients. We demonstrated several factors independently associated with aPTT prolongation. Monitoring aPTT levels may be indicated for select patients on LDUH thromboprophylaxis who are at high risk or consequence of bleeding and for aPTT prolongation.

A Retrospective Review of 193 Cases of Severe Eosinophilia in the Northeast United States.

BACKGROUND: This retrospective review evaluated the causes of severe eosinophilia (≥5000 eosinophils/L). Higher eosinophilia levels are more likely to cause tissue damage and may reflect disease severity. METHODS: We reviewed 193 cases of patients seen at Beth Israel Deaconess Medical Center in Boston, Massachusetts, and at the University of Vermont Medical Center in Burlington, Vermont, between January 2015 to May 2020 who had a peak absolute eosinophil count of at least 5000/µL. RESULTS: Thirty-nine percent of cases were attributable to a hematologic or oncologic cause. These cases had the highest mean peak absolute eosinophil count at 11,698/µL. Twenty percent of cases were secondary to drug reactions, of which 90% took place in an inpatient setting. Three percent of cases were from helminthic infection, the majority of which were in returning travelers. CONCLUSIONS: In our region of study, hematologic and oncologic cases are important causes of severe eosinophilia, drug reactions are a common etiology in the inpatient setting, and infections are a rare cause.

Things We Do for No Reason™: Card Flipping Rounds.

Inspired by the ABIM Foundation's Choosing Wisely® campaign, the "Things We Do for No Reason™" (TWDFNR) series reviews practices that have become common parts of hospital care but may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent "black and white" conclusions or clinical practice standards but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion.